United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
I nilid Stall-, Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



I0/5H4.44H 



00/22/2000 



2292 7590 10/16/2009 

BIRCH STEWART KOLASCH & BIRCH 
PO BOX 747 

FALLS CHURCH, VA 22040-0747 



1190-0627PUS1 



TEJANO, DWIGHT ALEX C 



PAPER NUMBER 



NOTIFICATION DATE | DELIVERY MODE 
10/16/2009 ELECTRONIC 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 

Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the 

following e-mail address(es): 

mailroom@bskb.com 



PTOL-90A (Rev. 04/07) 



l/ffflrC? nVrliUli Otfff Iff ids y 


Application No. 

10/584,448 


Applicant(s) 

MAKITA ET AL. 


Examiner 

Dwight Alex C. Tejano 


Art Unit 

2622 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 25 June 2009 . 
2a )^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 21-40 is/are pending in the application. 

4a) Of the above claim(s) 35 and 40 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |EI Claim(s) 21-26,33.34,38 and 39 is/are rejected. 

7) [3 Claim(s) 27-32.36 and 37 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 22 June 2006 & 15 Feb 2007 is/are: a)^ accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)E| All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.^ Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1) ^| Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 ) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26 d (Rev e 08-06r 



Office Action Summary 



Part of Paper No./Mail Date 20091012 



Application/Control Number: 10/584,448 
Art Unit: 2622 



Page 2 



DETAILED ACTION 

Election/Restrictions 
Claims 35 and 40 are withdrawn from further consideration pursuant to 37 
CFR 1.142(b) as being drawn to a nonelected species, there being no allowable generic 
or linking claim. Election was made without traverse in the reply filed on 25 June 2009. 

Response to Arguments 
Applicant's arguments with respect to claims 21 - 26 have been considered but 
are moot in view of the new ground(s) of rejection. 

Claim Objections 
Claims 31, 32, and 36 - 39 are objected to because of the following 
informalities: 

• Claims 31 , 32: The instances of the word "obtains" should be "obtain." 

• Claims 36 - 39: The preambles all refer back to the apparatus according 
to claims 24 and 25, respectively. However, claims 24 and 25 do not 
claim an apparatus, but rather a signal processing method. 

Appropriate correction is required. 
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Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



Claims 24, 25, 38, and 39 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Zhang, et al. (US 20040086177 A1.) 

Regarding claim 24, Zhang, et al. (hereafter, "Zhang") discloses a pixel signal 
processing method (Fig. 8) that, given a group of pixel signals from pixels (25) arrayed 
on a two-dimensional plane (sensor, 20), each pixel having one of a first to an N-th 
spectral characteristic (RGB [0005]), generates a pixel signal having an L-th spectral 
characteristic at a pixel position of interest where there is a pixel signal having a K-th 
spectral characteristic (K and L being different integers between 1 and N, inclusive), 
comprising (output demosaic, 890): 

• selecting one of the arrayed pixels for the pixel position of interest 
(inherent, as illustrated in that specific green pixel positions are "selected", 
[0059]); 

• applying low-pass filtering (Gaussian convolution, which effectively low- 
pass filters image, [0063]) to a plurality of pixel positions in an area 
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neighboring the pixel position of interest (blurred version of regions, 830) 
according to the K-th spectral characteristic and the L-th spectral 
characteristic; 

• performing a regression analysis in the plurality of pixel positions in the 
area neighboring the pixel position of interest (calculate regression 
coefficients, 850), using the pixel signals or the low-pass filtered outputs 
(from blurred versions of regions, which only contain high frequency 
components, [0064]) having the K-th spectral characteristic as an 
explanatory variable, and using the pixel signals or the low-pass filtered 
outputs having the L-th spectral characteristic as a purpose variable, to 
calculate a regression line, y = ax + b [equation, 0054] 

(V being the pixel signal having the L-th spectral characteristic, 'x' being 
the pixel signals having the K-th spectral characteristic, 'a' and 'b' being 
constants representing the slope and intercept of the regression line) 
[0054,] 

• expressing a correlation of the pixel signals having the K-th spectral 
characteristic with the pixel signals having the L-th spectral characteristic 
(Fig. 2) [0057]; and 

• determining the pixel signal having the L-th spectral characteristic at the 
pixel position of interest by applying a conversion formula based on the 
regression line to the pixel signal having the K-th spectral characteristic at 
the pixel position of interest [0057]; 
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• wherein the selecting step sequentially selects different ones of the 
arrayed pixels for the pixel position of interest and, for each selected pixel 
position of interest, applying the steps of performing the regression 
analysis and determining the pixel signal having the L-th spectral 
characteristic [process is repeated for all others, 0061 .] 

Regarding claim 25, Zhang discloses the limitations of claim 24 and further that 
the pixel signals (raw sensor values, 30) are associated with an imaging device (sensor, 
20) with N types (Bayer filter, [0005]) of photoelectric conversion elements (pixels, 25), 
each having one of the first to N-th spectral characteristics [0005], arrayed on a two- 
dimensional plane, wherein the method further comprises determining the K-th and L-th 
spectral characteristics in order of the strength of the correlation between their spectral 
characteristics [yielding highest correlation is selected, 0059.] 

Regarding claim 38, Zhang meets the limitations of claim 25. Furthermore, 
Zhang discloses that low-pass filtering (Gaussian convolution) is performed on an 
output of the imaging device (sensor, 20.) Specifically, the Gaussian convolution is 
performed on the raw image data, which constitutes low-pass filtering on the output data 
from the imaging device [0063.] 

Regarding claim 39, Zhang satisfies the limitations of claim 25. Additionally, the 
regression analysis must inherently use the pixel signals output from the imaging device 
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as the pixel signals having the K-th spectral characteristic on account of the fact that 
there are no other sources for pixel signal data for the process to analyze. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 21, 22, 23, 26, 33, and 34 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Zhang. 

Regarding claim 21, Zhang discloses a pixel signal processing apparatus (Fig. 3) 
that, given a group of pixel signals (raw sensor values, 30) from pixels (25) arrayed on a 
two-dimensional plane (sensor, 20), each pixel having one of a first to an N-th spectral 
characteristic [0005], generates a pixel signal having an L-th spectral characteristic at a 
pixel position of interest where there is a pixel signal having a K-th spectral 
characteristic (K and L being different integers between 1 and N, inclusive.) 

Additionally, Zhang discloses a regression analysis device and calculating device 
(processor, 40, with LLR algorithm, 45) that performs the regression calculations and 
conversion calculations as required by the claim. 

However, Zhang fails to specifically disclose a selector for selecting a pixel and a 
low-pass filter corresponding to the different spectral characteristics. 
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Despite this, the Examiner maintains that it would have been obvious to include 
such things, given the nature of Zhang's system. 

First, because Zhang discloses the selections of pixels to perform regression on 
or to divide into regions, it would be obvious to include a selector, as it is necessary to 
be able to even perform this step. 

Additionally, Zhang discloses a software method of retrieving a low-pass filtered 
version of the image (using Gaussian convolution.) Because Zhang discloses that this 
filtered image is preferable in order to be able to extract the high frequency components 
of the image, it would have further been obvious to anyone of ordinary skill in the art 
that this step could alternatively be performed by hardware instead of software. This is 
mainly a design choice (additional software processing time vs. additional hardware 
components,) as the ultimate effect is, predictably, the same in either case. 

Therefore, it would be obvious to include the selector and the low-pass filter to 
Zhang's system, as the selector is necessary for the method in Zhang to perform 
properly and the hardware based low-pass filters would eliminate the processing time 
caused by the Gaussian convolution. 

Regarding claim 22, Zhang renders the apparatus of claim 21 obvious. 
Furthermore, because the selector is obviously necessary on account of the method as 
required by Zhang, it would further be obvious that the selector must also determine the 
spectral characteristics in order of strength, as, again, the method of Zhang requires it. 
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Because Zhang discloses a selection based on determination of correlation strength, 
the selector must obviously have the same properties. 

Therefore, it would be obvious that the selector determines the characteristics in 
order of strength of the correlation, as the method in Zhang requires this component in 
order for it perform the corresponding process [0059.] 

Claims 33 and 34 are apparatus variants of claims 38 and 39, respectively. 
They are thus rendered obvious for the same rationale as in the rejection to claim 22 
(i.e., apparatus is rendered obvious as being required to perform the disclosed method). 

Regarding claim 26, Zhang discloses the process of claim 25, as discussed 
above. Additionally, Zhang discloses that the imaging device has one of red, green, 
and blue spectral characteristics [0005.] 

However, Zhang fails to disclose the selection means as determining the specific 
order as claimed of pixel signals for comparison (specifically, G-R, G-B, R-G, B-G, B-R, 
R-B.) Despite this, such a practice is obvious to one of ordinary skill, as this process is 
mainly a design choice as to what to process first. 

Zhang discloses in [0054] by example that the G-R and G-B relationships are 
evaluated first; that is, determining the placement of the missing R and B values in a 
selected G pixel. Further, Zhang discloses that the "similar estimations can be applied 
for all missing R and G values at the B pixel locations" and further for the R locations. 
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Because the process is evaluated with respect to each pixel individually (i.e., the 
neighborhood for one pixel is not the same as the neighborhood for another pixel) and 
is done with every color relationship (R-G, R-B, B-G) with separate linear regression 
analyses, the outcome of one evaluation would not change the outcome of another 
evaluation. As such, the color relationships can be evaluated in any order in the 
process without changing the outcome of each evaluation. 

Therefore, the claimed process would have been obvious to one of ordinary skill 
in the art because the particular process is immaterial compared to the evaluation of 
another pair of pixel signals and, as such, was recognized as part of the ordinary 
capabilities with highly predictable results of one skilled in the art. 

Regarding claim 23, Zhang renders the apparatus of claim 21 obvious. In a 
similar fashion, the apparatus of claim 23 is rendered obvious in that, because the 
analogous process (claim 26) is rendered obvious by Zhang, the apparatus that is 
necessary to perform the process is also obvious in view of Zhang. 

Therefore, this apparatus variation is rendered obvious, as the apparatus 
components are necessary to perform the process of claim 26, as rendered obvious for 
the rationale as previously presented. 
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Allowable Subject Matter 
Claims 27 - 32, 36, and 37 objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all 
of the limitations of the base claim and any intervening claims. 

The prior art of record fails to disclose or reasonably suggest the limitations of 
the above claims. Specifically, the prior art of record fails to specifically disclose or 
reasonably suggest the determination processes as required by claims 27 and variant 
claim 29 and the specific low-pass filter limitations required by claims 28 - 32, 36, and 
37. In these cases, while a selector and a low-pass filter are themselves rendered 
obvious, as discussed above, these limitations are otherwise unique. 

Citation of Pertinent Art 

The prior art made of record is considered pertinent to the applicant's disclosure, 
but is not relied upon as a reference for the preceding sections: 

• Jones (US 200201 63583 A1 ) discloses a camera with demosaicing properties 
in order to increase dynamic range. 

• Watanabe (US 20030227552 A1 ) discloses an imaging device that divides 
into pixel neighborhoods and interpolates accordingly. 

• Baharav, et al. (US 20040201 721 A1 ) discloses a system for demosaicing 
and resizing images. 

• Keshet, et al. (US 20050134713 A1) discloses a method for interpolating 
color values in a pixel neighborhood. 
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• Kondo, et al. (US 20050200723 A1 ) discloses an image processing apparatus 
that performs ADRC processes in pixel demosaicing. 

• Sasai (US 6654492 B1 ) discloses an image processing apparatus that has an 
interpolation unit and has an average of values in a neighborhood of pixels. 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dwight Alex C. Tejano whose telephone number is (571) 
270-7200. The examiner can normally be reached on Monday through Friday 10:00- 
6:00 with alternate Fridays off. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jason Chan can be reached on (571) 272-3022. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Lin Ye/ 

Supervisory Patent Examiner, Art Unit 2622 



/Dwight Alex C Tejano/ 
Examiner, Art Unit 2622 



